Summary: A framework for learning complex structured output representations is presented. To this end variational auto-encoders (VAE) are extended to conditional VAEs,' i.e., conditioned on the input data x. Quality - The paper is mostly well written, could however be improved occasionally. Clarity - The idea is clearly presented but some details are missing. Originality - Conditional VAEs seem to be a straightforward extension of standard VAEs, but certainly worth a discussion Significance - The significance could be improved by a more extensive evaluation showing results for various modifications Comments: - I think the term generative is typically used when learning distributions that also involve the input data.

Given that the method is only approximate and forgetting a specific data point cannot be theoretically guaranteed, can the authors comment on how practically applicable the proposed approach is? Or are the GDPR requirements so strict as to require retraining/proofed forgetting to be fulfilled making the paper a nice first step, but leaving lots of further problems until the formal requirements are met? - l67 and others refer to the Kullback Leibler divergence as a distance. Given that it is not a distance due to its lack of symmetry it should properly be called divergence or relative entropy.

Summary: The authors address the problem of learning category-specific shape reconstruction using the proposed NOX representation. The NOX representation builds on the NOCS idea of normalized object coordinate systems which represents all instances in an object category within a unit cube. Predicting a perspective projection of the NOCS representation in the camera view (called the NOCS map) is thus equivalent to predicting the object shape coordinates in the unit cube (or NOCS). The authors extend this to not just predict NOCS coordinates of the visible surface in a camera view (first intersection of ray from pixel to object) but also coordinates of the the "last* intersection of the ray. This pair of first and last intersection maps termed NOX thus provide a reasonably complete picture of object shape (for mostly convex objects).

Update after rebuttal: Due to author comments and, in particular, discussions with the other reviewers, I have updated my score from 4 to a weak accept 6. For the future draft, aside from the revisions and clarifications the authors have promised in the rebuttal, I recommend the following (slight) modifications to improve the manuscript: The motivation in the introduction would be strengthened by drawing clearer connections to the real world. The authors should consider picking a specific real world example and illustrating the method through that example (even if it's not possible to provide simulation results on such an example). In line with this, the authors should be careful about discussion of safe-RL. Typically such methods involve use of constraints to ensure safety, but it does not appear the authors explicitly use or discuss such methods here.

Adding experimental results as they promised to R3 would be valuable as well 3) As R2 pointed out, the intuition behind the analysis is not always clear. Given the rather convincing answers in the rebuttal, I think the authors can easily improve this aspect in the revised version.